Method and apparatus with iris region extraction

ABSTRACT

A method and apparatus for extracting an iris region is disclosed. The apparatus may generate a classification map associated with an iris region from an eye image using a trained neural network model, estimate a geometric parameter associated with the iris region using the generated classification map, and extract the iris region from the eye image based on the estimated geometric parameter.

CROSS-REFERENCE TO RELATED APPLICATION(S)

This application claims the benefit under 35 USC § 119(a) of KoreanPatent Application No. 10-2016-0150488 filed on Nov. 11, 2016, in theKorean Intellectual Property Office, the entire disclosure of which isincorporated herein by reference for all purposes.

BACKGROUND 1. Field

The following description relates to methods and apparatuses thatinclude iris region extracting technology.

2. Description of Related Art

Recently, interest in technology for verifying an identity of a userusing a biometric feature has been increasing. Biometric authenticationtechnology, such as, for example, facial recognition, fingerprintrecognition, vein pattern recognition, and iris recognition, is used toverify an identity of a user using a unique biometric feature differentfrom individual to individual. While facial recognition and thefingerprint recognition are widely used, iris recognition has beengaining attention.

Iris recognition is a type of contactless recognition method, and isused to recognize a user by analyzing an iris pattern of the user. Irispatterns may differ between a left eye and a right eye of a same user,and differ between identical twins having a same genetic structure, butotherwise iris patterns may not greatly vary over time. In irisrecognition, an iris region distinguished from a pupil and a sclera maybe extracted from an image that includes a user's eye(s), and featuresmay be extracted from the extracted iris region. Such features may becompared to registered features and the user may be authenticated if theresults of the comparisons are within a threshold, for example.

SUMMARY

This Summary is provided to introduce a selection of concepts in asimplified form that are further described below in the DetailedDescription. This Summary is not intended to identify key features oressential features of the claimed subject matter, nor is the Summaryintended to be used as an aid in determining the scope of the claimedsubject matter.

In one general aspect, a processor implemented iris region extractionmethod includes obtaining an eye image, extracting an iris region fromthe obtained eye image using a trained neural network model.

The method may further include obtaining an image that includes an eyeand performing a cropping of the image to obtain the eye image,extracting features from the extracted iris region, and matching theextracted features to registered iris information.

The trained neural network may be a neural network with one or moreconvolutional hidden layers, and may be trained based on abackpropagation method that uses training eye image information that islabeled with respect to different classifications for at leastcorresponding pupil and iris regions or labeled with respect tocorresponding pupil and iris geometric parameters.

The extracting of the iris region may include performing a firstsegmentation of the eye image by providing a lower resolution image ofthe eye image to a first trained neural network, performing a secondsegmentation of the eye image using a second trained neural network, thesecond segmentation of the eye image being dependent on results of thefirst segmentation of the eye image, and extracting the iris region fromthe eye image based on results of the second segmentation.

The extracting of the iris region may include extracting pixelscorresponding to the iris region from the eye image as an output of thetrained neural network model.

The extracting of the iris region may include generating a firstclassification map associated with the iris region from the eye imageusing the trained neural network model, estimating one or more geometricparameters associated with the iris region using the generated firstclassification map, and extracting the iris region from the eye imagebased on the estimated one or more geometric parameters.

The estimating of the one or more geometric parameters may includeperforming a fitting operation of one or more geometric equations to thefirst classification map, the fitting may include estimating geometricparameters of at least one of a circle, ellipse, or a curve that arefitted to features of the first classification map.

The one or more geometric parameters may include plural geometricparameters, including a first parameter to define a shape of a firstcircle or a first ellipse corresponding to an outer boundary of the irisregion represented in the first classification map and a secondparameter to define a shape of a second circle or a second ellipsecorresponding to a boundary between the iris region and a pupil regionrepresented in the first classification map.

The plural geometric parameters may further include at least one of athird parameter to define a shape of a first curve corresponding to anupper eyelid or a fourth parameter to define a shape of a second curvecorresponding to a lower eyelid.

The extracting of the iris region may include generating a secondclassification map based on the estimated one or more geometricparameters, and extracting the iris region from the eye image using thegenerated second classification map.

The extracting of the iris region may include generating alower-resolution image of the eye image, generating a classification mapassociated with the iris region from the lower-resolution image usingthe trained neural network model, estimating one or more geometricparameters associated with the iris region using the generatedclassification map, and extracting the iris region from the eye imagebased on the estimated one or more geometric parameters.

The generating of the lower-resolution image of the eye image may beperformed by changing a resolution of the eye image to generate thelower-resolution image.

The estimating of the one or more geometric parameters may includeadjusting a size of the classification map to match a size of the eyeimage, and estimating the one or more geometric parameters using theadjusted size classification map.

The extracting of the iris region may include generating alower-resolution image of the eye image, generating a classification mapassociated with the iris region from the generated lower-resolutionimage using a first trained neural network model, determining aplurality of refinement regions in the eye image using the generatedclassification map, extracting pixels from the refinement regions usinga second trained neural network, and extracting the iris region from theeye image based on a result of the extracting of the pixels.

The determining of the refinement regions may include determining therefinement regions in the eye image based on the classification map andstructural information predefined with respect to iris regions.

The extracting of the iris region may include estimating one or moregeometric parameters associated with the iris region based on the resultof the extracting of the pixels, and extracting the iris region from theeye image based on the estimated one or more geometric parameters.

The extracting of the iris region may include generating alower-resolution image of the eye image, obtaining a geometric parameterassociated with the iris region from the generated lower-resolutionimage using a first trained neural network model, determining aplurality of refinement regions in the eye image using the obtainedgeometric parameter, extracting pixels from the refinement regions usinga second trained neural network model, and extracting the iris regionfrom the eye image based on a result of the extracting of the pixels.

The extracting of the iris region may include estimating one or moregeometric parameters associated with the iris region based on the resultof the extracting of the pixels, and extracting the iris region from theeye image based on the estimated one or more geometric parameters.

The extracting of the iris region may include obtaining a geometricparameter associated with the iris region from the eye image using thetrained neural network model, and extracting the iris region from theeye image based on the obtained geometric parameter.

The method may further include respectively analyzing a first eye image,which is a captured color image of an eye of a user, and a second eyeimage, which is a captured infrared image of the eye of the user, andselecting one of the first eye image and the second eye image to be theobtained eye image.

The obtaining of the eye image may include extracting a region ofinterest (ROI), including the iris region, from the input image, as theobtained eye image.

In one general aspect, one or more embodiments include a non-transitorycomputer-readable storage medium storing instructions that, whenexecuted by a processor, cause the processor to perform one or more orall operations described herein.

In one general aspect, an apparatus includes one or more processorsconfigured to extract an iris region from an obtained eye image using atrained neural network model.

The apparatus may further include at least one camera controlled by theone or more processors to obtain the eye image.

The trained neural network may be a neural network with one or moreconvolutional hidden layers, and may be trained based on abackpropagation method that uses training eye image information that islabeled with respect to different classifications for at leastcorresponding pupil and iris regions or labeled with respect tocorresponding pupil and iris geometric parameters.

The processor may be configured to perform the training of the neuralnetwork model.

The apparatus may further include a memory that may store the trainedneural network model.

The processor may be configured to generate a classification mapassociated with the iris region from the eye image using the trainedneural network model, estimate one or more geometric parametersassociated with the iris region using the generated classification map,and extract the iris region from the eye image based on the estimatedone or more geometric parameters.

To estimate the one or more geometric parameters, the one or moreprocessors may be configured to perform a fitting operation of one ormore geometric equations to the classification map, and the fitting mayinclude estimating geometric parameters of at least one of a circle,ellipse, or a curve that are fitted to features of the classificationmap.

The one or more geometric parameters may include plural geometricparameters, including a first parameter to define a shape of a firstcircle or a first ellipse corresponding to an outer boundary of the irisregion represented in the classification map and a second parameter todefine a shape of a second circle or a second ellipse corresponding to aboundary between the iris region and a pupil region represented in theclassification map.

The processor may be configured to generate a lower-resolution image ofthe eye image, generate a classification map associated with the irisregion from the generated lower-resolution image using a first trainedneural network model, with the classification map having a lowerresolution than the eye image, determine a plurality of refinementregions in the eye image using the generated classification map, extractpixels from the refinement regions using a second trained neural networkmodel, and extract the iris region from the eye image based on a resultof the extracting of the pixels.

The processor may be configured to generate a lower-resolution image ofthe eye image, obtain a geometric parameter associated with the irisregion from the generated lower-resolution image using a first trainedneural network model, determine a plurality of refinement regions in theeye image using the obtained geometric parameter, extract pixels fromthe determined plural refinement regions using a second trained neuralnetwork model, estimate one or more geometric parameters associated withthe iris region based on a result of the extracting of the pixels, andextract the iris region from the eye image based on the estimated one ormore geometric parameters.

In one general aspect, a processor implemented iris region extractionmethod includes providing a first image for an eye to a first trainedneural network to generate an output of the first trained neuralnetwork, estimating one or more geometric parameters by performing afitting operation of one or more geometric equations using the output ofthe first trained neural network, the fitting including estimatinggeometric parameters of at least one of a circle, ellipse, or a curvefor an iris region of the eye, and extracting the iris region from asecond image for the eye based on the estimated one or more geometricparameters.

The extracting of the iris region may include determining a plurality ofrefinement regions in the second image based on the estimated one ormore geometric parameters, extracting pixels from the refinement regionsusing a second neural network model, and extracting the iris region fromthe second image based on a result of the extracting of the pixels.

The extracting of the iris region may include estimating at least onegeometric parameter associated with the iris region based on the resultof the extracting of the pixels, and extracting the iris region from thesecond image based on the estimated at least one geometric parameter.

The one or more geometric parameters may include plural geometricparameters, including a first parameter to define a shape of a firstcircle or a first ellipse corresponding to an outer boundary of the irisregion represented in the output of the first trained neural network anda second parameter to define a shape of a second circle or a secondellipse corresponding to a boundary between the iris region and a pupilregion represented in the output of the first trained neural network.

The first image may be an infrared image. The second image may be acolor image.

Other features and aspects will be apparent from the following detaileddescription, the drawings, and the claims.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram illustrating an example of an iris recognitionapparatus.

FIG. 2 is a flowchart illustrating an example iris region extractionmethod.

FIG. 3 is a diagram illustrating an example iris region extractionmethod.

FIG. 4 is a flowchart illustrating an example iris region extractionmethod.

FIG. 5 is a diagram illustrating an example iris region extractionmethod.

FIG. 6 is a flowchart illustrating an example iris region extractionmethod.

FIG. 7 is a diagram illustrating an example iris region extractionmethod.

FIG. 8 is a flowchart illustrating an example iris region extractionmethod.

FIG. 9 is a diagram illustrating an example iris region extractionmethod.

FIG. 10 is a flowchart illustrating an example iris region extractionmethod.

FIG. 11 is a diagram illustrating an example apparatus configured toperform iris region extraction according to one or more embodiments.

FIG. 12 is a diagram illustrating an example computing apparatusconfigured to perform iris region extraction according to one or moreembodiments.

Throughout the drawings and the detailed description, unless otherwisedescribed or provided, the same drawing reference numerals will beunderstood to refer to the same or like elements, features, andstructures. The drawings may not be to scale, and the relative size,proportions, and depiction of elements in the drawings may beexaggerated for clarity, illustration, and convenience.

DETAILED DESCRIPTION

The following detailed description is provided to assist the reader ingaining a comprehensive understanding of the methods, apparatuses,and/or systems described herein. However, various changes,modifications, and equivalents of the methods, apparatuses, and/orsystems described herein will be apparent after an understanding of thedisclosure of this application. For example, the sequences of operationsdescribed herein are merely examples, and are not limited to those setforth herein, but may be changed as will be apparent after anunderstanding of the disclosure of this application, with the exceptionof operations necessarily occurring in a certain order. Also,descriptions of features that are known in the art may be omitted forincreased clarity and conciseness.

The features described herein may be embodied in different forms, andare not to be construed as being limited to the examples describedherein. Rather, the examples described herein have been provided merelyto illustrate some of the many possible ways of implementing themethods, apparatuses, and/or systems described herein that will beapparent after an understanding of the disclosure of this application.

The terminology used herein is for the purpose of describing particularexamples only, and is not to be used to limit the disclosure. As usedherein, the singular forms “a,” “an,” and “the” are intended to includethe plural forms as well, unless the context clearly indicatesotherwise. As used herein, the term “and/or” includes any one and anycombination of any two or more of the associated listed items. As usedherein, the terms “include,” “comprise,” and “have” specify the presenceof stated features, numbers, operations, elements, components, and/orcombinations thereof in at least one embodiment, such as when it isindicated that such stated features, numbers, operations, elements,components, and/or combinations thereof may be included in an example.However, the use of the terms “include,” “comprise,” and “have” in theSpecification do not preclude the presence or addition of one or moreother features, numbers, operations, elements, components, and/orcombinations thereof in other embodiments, and do not preclude in theSpecification the lack of presence of any of such features, numbers,operations, elements, components, and/or combinations thereof in stillother embodiments unless explicitly or contextually/implicitly clearlyexplained otherwise.

In addition, terms such as first, second, A, B, (a), (b), and the likemay be used herein to describe components. Each of these terminologiesis not used to define an essence, order, or sequence of a correspondingcomponent but used merely to distinguish the corresponding componentfrom other component(s).

Throughout the specification, when an element, such as a layer, region,or substrate, is described as being “on,” “connected to,” or “coupledto” another element, it may be directly “on,” “connected to,” or“coupled to” the other element, or there may be one or more otherelements intervening therebetween. In contrast, when an element isdescribed as being “directly on,” “directly connected to,” or “directlycoupled to” another element, there can be no other elements interveningtherebetween. Likewise, expressions, for example, “between” and“immediately between” and “adjacent to” and “immediately adjacent to”may also be construed as described in the foregoing.

Unless otherwise defined, all terms, including technical and scientificterms, used herein have the same meaning as commonly understood by oneof ordinary skill in the art to which this disclosure pertainsconsistent with and after an understanding of the present disclosure.Terms, such as those defined in commonly used dictionaries, are to beinterpreted as having a meaning that is consistent with their meaning inthe context of the relevant art and the present disclosure, and are notto be interpreted in an idealized or overly formal sense unlessexpressly so defined herein.

A function or an operation illustrated in a block may be performed notin a sequential order according to examples. For example, functions oroperations illustrated in successive blocks may be actually performedconcurrently, or an order of the blocks may be changed based on relatedfunctions or operations.

FIG. 1 is a diagram illustrating an example of an iris recognitionapparatus.

Herein, iris recognition refers to biometrics technology that, dependingon embodiment, may include performing an authentication of a user usinga pattern of an iris present between a pupil in the middle of an eye ofthe user and a sclera of the user. In differing embodiments, the irisrecognition may include, for example, authentication of a user login,mobile payment, and access control. The iris recognition may includeobtaining an eye image and recognizing an iris in the obtained eyeimage, and the iris recognition may also include the obtaining of theeye image, the recognizing of the iris, and the performing of one ormore such authentication operations. The recognizing of the iris mayinclude extracting an iris region, extracting a feature of the iris, andperforming a matching of the extracted feature with one or moreregistered iris' features, for example.

Referring to FIG. 1, a computing apparatus 110 performs anauthentication operation for a user 120 trying to access the computingapparatus 110 through iris recognition. For example, the user 120 maymake an attempt at iris recognition-based user authentication in thecomputing apparatus 110 to unlock the computing apparatus 110, passwordentry for applications executed by the computing apparatus 110, orauthorizing payments or other financial transactions by authenticatingthe user as an authorized person for such payments or transactions. Theiris recognition may also be implemented for one or more users, such asmultiple people captured in a social photograph, which may be capturedby the camera of the computing apparatus 110, such as for automatedcategorizing or identification of users in an executed photo album orsocial networking application of the computing apparatus 110, forexample. The computing apparatus 110 may obtain an eye image includingan eye region of the user 120 using an image acquirer, for example, aninfrared camera and a charge-coupled device (CCD) camera, and analyze aniris pattern of an iris in the obtained eye image to allow or reject theuser authentication, i.e., as a result of the authentication operation.The camera may also be a color camera or gray scale camera, or thecaptured image may selectively be a color image, a single color image, agray scale image, etc., and the computing apparatus may include multiplesuch different cameras. The computing apparatus 110 may compare afeature of the iris pattern in the eye image to a registered irispattern feature stored in a non-transitory memory of the computingapparatus 110, or otherwise made available to or accessed by thecomputing apparatus 110, and the computing apparatus 110 may determinewhether to authenticate the user 120 based on a result of the comparing.In any of the described examples, the extracted iris may also be storedin a memory of the computing apparatus 110, and may also be provided toan external device that may perform the authentication operation. Thecomputing apparatus 110 may also be configured to register an irisextracted to any of the described examples herein, e.g., for asubsequent use for authentication comparisons with other extractedirises. The computing apparatus 110 may be, for example, a smartphone, awearable device, a tablet computer, a netbook, a laptop, a desktop, apersonal digital assistant (PDA), a set-top box, a home appliance, anautomated teller machine (ATM), a door lock or door/area entry/accesssystem, a security device, and a vehicle starting device, and may beconfigured as illustrated in any FIGS. 1, 11, and 12, for example. Asonly an example, an embodiment where the computing apparatus 110 is asmartphone is illustrated in FIG. 1.

In one example, in a case that the computing apparatus 110 operates in alocked mode, e.g., preventing access to other functional operations ofthe computing apparatus 110, the computing apparatus 110 may becontrolled, or automatically perform, an authentication operation of theuser 120 through iris recognition. When the user 120 is allowed, e.g.,if the computing apparatus is configured to permit iris authentication,the user 120 may unlock the computing apparatus 110 through the irisrecognition performed by the computing apparatus 110. The irisrecognition may be performed upon user interaction or manipulation ofthe computing apparatus, or the recognition may be automaticallyperformed. The user 120 may register or store, in advance, biometricinformation associated with an iris of the user 120 in the computingapparatus 110, and the computing apparatus 110 may store the biometricinformation associated with the iris of the user 120 in a storage or acloud storage. Embodiments include such registering and storing of thebiometric information associated with one or both of the irises of theuser 120. When the user 120 presents an eye of the user 120 to the imageacquirer of the computing apparatus 110 to unlock the computingapparatus 110, the computing apparatus 110 may extract a feature of theiris indicated in an iris pattern from an image including an eye regionof the user 120, and verify whether the extracted feature matches aregistered feature. When the extracted feature matches the registeredfeature, the computing apparatus 110 may cancel the lock mode and allowaccess to the user 120 of further functions of the computing apparatus110. Conversely, when the extracted feature does not match theregistered feature, the computing apparatus 110 may maintain the lockmode and restrict the user 120 from accessing additional functions ofthe computing apparatus 110 or from accessing data stored in thecomputing apparatus 110.

To extract the feature of the iris of the user 120 by the computingapparatus 110, the computing apparatus 110 may be configured to extractan iris region from the image, noting that embodiments are not limitedto the computing apparatus 110 and other computing apparatuses may beconfigured to extract the iris region, depending on embodiment. Thus,hereinafter, such computing apparatuses may simply be referred to as aniris region extracting apparatus. In addition, in one example, such aniris region extracting apparatus may be a component of the computingapparatus 110, or represented by the computing apparatus 110 itself, andextract the iris region from an eye image or a face image obtained bythe example image acquirer of the computing apparatus 110 or providedfrom an external image acquirer. For example, in one or moreembodiments, the iris region extracting apparatus may extract, moreaccurately and rapidly, the iris region from the image using a neuralnetwork model trained based on training data, e.g., compared to previousapproaches, such as where edge detection operations are performed andthe iris region boundaries being identified based on the detected edges.Herein, the iris extraction methods may be performed for either or bothof the registration of irises and/or iris features and forauthentication operations that compare features of extracted irisregions to previously registered iris features, including such alternatemethods being performed by the same or different apparatuses. A methodof extracting an iris region, hereinafter simply referred to as an irisregion extracting method, which may be performed by such an example irisregion extracting apparatus, noting that embodiments are not limitedthereto.

FIG. 2 is a flowchart illustrating an example iris region extractionmethod.

Referring to FIG. 2, in operation 210, an iris region extractingapparatus obtains an eye image. For example, the iris region extractingapparatus may capture the eye image using a camera or receive the eyeimage from an external source. The eye image refers to an imageincluding at least an eye region of a user and may be, for example, acolor image, a gray image, and an infrared image. In an example, theiris region extracting apparatus may receive, as an input image, atleast a face image including a face region or a face image including aportion of the face region, and obtain an eye image by extracting aregion of interest (ROI) including an iris region from the input image.

In an example, the iris region extracting apparatus may obtain a firsteye image, which is a captured color image of an eye of a user, and asecond eye image, which is a captured infrared image of the eye of theuser, and selects a target image from which an iris region is to beextracted from the obtained first eye image and the obtained second eyeimage. For example, in a case that an eye region in the first eye imageis determined to include an artifact, for example, a light blur, theiris region extracting apparatus may select the second eye image as thetarget image from which the iris region is extracted. For anotherexample, in a case that the first eye image is determined to have agreater quality than the second eye image, the iris region extractingapparatus may select the first eye image as the target image from whichthe iris region is extracted. In still another example, the iris regionextracting apparatus may perform iris extraction discussed herein forboth example eye images. Here, though the first and second eye imagesare referred to as being different types of captured images, e.g., colorversus infrared, alternatively the example first and second eye imagesmay be both color images or both infrared images, and the first andsecond eye images may be obtained or captured at a same time or atdifferent times.

In operation 220, the iris region extracting apparatus extracts an irisregion from the obtained eye image using a neural network model, or adeep neural network model. For example, compared to a neural networkmodel, a deep neural network model may be a neural network that has morethan one hidden layer. The neural network model may be configured as aclassifier to provide information to be used to determine the irisregion based on image information of the eye image input to the neuralnetwork model. The image information of the eye image may be informationassociated with a pixel value of each of pixels included in the eyeimage, for example, a color value and a brightness value, and aninfrared detection value in a case of the eye image being the infraredimage.

An output of the neural network model may be a geometric parameter orgeometric parameters that can be used to extract or classify pixelscorresponding to the iris region from the eye image or define the irisregion. In such examples, the neural network model may provide orgenerate a classification map in which the iris region is distinguishedfrom the eye image, or may output a parameter(s) of a circle or curveequations that can be considered or used as representations of ordefining a boundary of the iris region. For example, for generating oridentifying the geometric parameters of the pupil and iris, the neuralnetwork may be a regression-based neural network, which includes one ormore convolutional neural network hidden layers followed by one or morefully connected neural network hidden layers. In such an example, anembodiment includes training the neural network using labeled traininginput images, e.g., for which pupil and iris parameters are known. Theclassification map may be a map or an image indicating which class orregion a pixel of the eye image belongs to. For example, theclassification map may indicate which class or region, for example, apupil region, an iris region, and a background excluding the pupilregion and the iris region, a pixel of the eye image belongs to. In anexample, a corresponding classification-based neural network may includeone or more convolution neural network hidden layers followed by one ormore fully connected neural network hidden layers, for example, suchthat the classification map or mapping may be output. For example, aninput image patch may be provided to the classifying neural networkmodel and the neural network model may identify or output aclassification of a k-th pixel of the patch. In such an example, theclassification-based neural network model may be trained based onlabeled training input patch images, e.g., for which a classification ofcorresponding k-th pixels are known.

Thus, as an example of the neural network model, a neural network modelconfigured to classify a region by a pixel unit, calculate a parameterassociated with the region, or to output an image corresponding to theeye image may be used. For example, the neural network model describedin the foregoing may include a SegNet and a fully convolutional network(FCN) that is configured to output an image obtained as a result ofclassification from an input image through multiple convolution neuralnetwork hidden layers, or through one or more convolution and one ormore deconvolution neural network hidden layers. The output image mayalso include classification between the pupil, iris, and parts of theeye other than the pupil and the iris. Embodiments include the trainingof the fully convolutional neural network or theconvolutional-deconvolutional neural network, such as based on labeledinput eye images where such different classifications are known or theoutput classified image is known. Alternatively, as noted above, aneural network model configured to classify a k-th or central pixel of apartial patch image of an input image into each class or region, forexample, a pupil region, an iris region, or a background region may beused. In a case of using the partial patch image, pixel classificationmay be performed based on information on surrounding or neighborhoodregions, and thus accuracy in the pixel classification may be improvedand an accurate result of the pixel classification may be obtaineddespite large amounts of noise in the eye image and/or low contrast inthe eye image. Additionally, as only an example and as noted above, whenthe neural network model is configured to provide a geometricparameter(s) associated with a target that is to be classified in theinput image, for example, the iris region, the neuro network model maybe a regression-based neural network model.

The neural network model may be trained in advance through a training orlearning process based on a plurality of pieces of training data, ortrained through an on-device deep learning process performed in adevice, for example, the computing apparatus 110 of FIG. 1. For example,each of the differently described neural network models may berespectively trained based on the labeled input image information ordesired corresponding output images, classifications, or geometricparameters, such as through a backpropagation algorithm, e.g., whereconnection weightings between nodes of different hidden layers arerecursively adjusted until the corresponding neural network model istrained. The respectively trained neuro network may be stored in amemory of the iris region extracting apparatus. For example, the trainedneural network may be stored in a trained matrix format, where elementsof the matrix represent the corresponding trained weighted connectionsof the corresponding neural network structure. The stored trained neuralnetwork may further include hyper-parameter information, which maydefine the structure of the corresponding neural network for which theexample stored trained matrix correspond to. The hyper-parameters maydefine how many hidden layers, how many and respect structures of theconvolutional, deconconvolutional, and/or fully connected hidden layersof the neural network structure, the use of any bias and/or contextualnodes, and their structure within the neural network, and define anyrecurrent structures of the neural network, which may vary depending onembodiment.

Accordingly, in an example, the iris region extracting apparatus mayextract the iris region from the eye image based on a classification mapor geometric parameter(s) obtained using one or more neural networkmodels. Subsequent to operation 220, an iris feature, for example, aniris code, may be determined through normalization and filteringperformed on the extracted iris region, and a result of userauthentication may be determined based on a determined similaritybetween the determined iris feature and a registered iris feature(s),e.g., of authorized or previously authenticated users. Thus, the irisregion extracting apparatus may extract, more rapidly and accurately,the iris region from the eye image using such neural network models. Inaddition, in one or more embodiments, any one or multiple such neuralnetworks may be respectively used for different stages of the irisregion extraction, such as with one neural network configuration beingimplemented for a first segmentation operation and a second neuralnetwork configuration being implemented for a second segmentationoperation that is dependent on results of the first segmentationoperation.

Hereinafter, various examples of processes of extracting an iris regionby an iris region extracting apparatus using one or more of suchdescribed example neural network models implemented in operation 220will be described in greater detail below, with example implementationsof operation 220 being demonstrated in each of FIGS. 3-10, described ingreater detail below.

FIG. 3 is a diagram illustrating an example iris region extractionmethod.

Referring to FIG. 3, the iris region extracting apparatus extracts anROI 320 from an input image 310. The ROI 320 is a partial region of theinput image 310 that includes a pupil region 314 and an iris region 312.The iris region extracting apparatus extracts an iris region 354 from animage 330 of the ROI 320 using a neural network model, such as any ofthe aforementioned neural networks or combination of neural networksdescribed with respect to FIG. 2. Here, the image 330 is referred to asan eye image 330. In one example, image information of the eye image 330may be input to the neural network model, and geometric parameters todefine a shape of an iris region may be determined based on an outputresult of the neural network model. For example, as illustrated in FIG.3, the geometric parameters include parameters to define a first circle342 corresponding to an outer boundary of the iris region, a secondcircle 344 corresponding to a boundary between the iris region and apupil region 352, a first curve 346 corresponding to an upper eyelid,and a second curve 347 corresponding to a lower eyelid. The iris regionextracting apparatus may extract, as an iris region 354 from the eyeimage 330, a region surrounded by the first circle 342, the secondcircle 344, the first curve 346, and the second curve 347. Thus, usingthe neural network model, the iris region extracting apparatus may, moreaccurately and rapidly, extract the iris region 354 from the eye image330.

There may be a situation where an iris region may not be occluded by anupper eyelid or a lower eyelid. In such a case, a parameter of at leastone of the first curve 346 or the second curve 347 may not bedetermined. Here, the iris region extracting apparatus may extract theiris region 354 from the eye image 330 only using parameters of thefirst circle 342 and the second circle 344, or additionally usingparameters of one of the first curve 346 and the second curve 347 inaddition to the parameters of the first circle 342 and the second circle344.

FIG. 4 is a flowchart illustrating an example iris region extractionmethod.

Referring to FIG. 4, in operation 410, the iris region extractingapparatus generates a classification map associated with an iris regionfrom an eye image using a neural network model, such as any of theaforementioned neural networks or combination of neural networksdescribed with respect to FIG. 2. In response to image information ofthe eye image being input to the neural network model, the neuralnetwork model may extract pixels from the eye image with respect totheir determined or inferred class or region by the neural networkmodel, and the classification map may be generated based on a result ofthe extracting of the pixels. For example, in response to imageinformation of each patch region of the eye image being input to theneural network model, the neural network model may determine or infer,i.e., based on the previous training of the neural network, which of aniris region, a pupil region, and a background region a central pixel ofeach patch region belongs to. The background region refers to a regionthat is not the iris region and not the pupil region, and may include asclera region and a region occluded by an eyelid. In response to theextracting being completed on all the pixels of the eye image, forexample, the classification map indicating which pixels of the eye imagebelong to each of the pupil region, the iris region, and the backgroundregion may be generated. The number of classes into which the pixels areclassified may vary depending on embodiment. As described above, in theclassification map, the pixels may be classified into three exampleclasses, for example, the iris region, the pupil region, and thebackground region, or into two classes, for example, the iris region anda non-iris region. For another example, the pixels may be classifiedinto four or more classes. Thus, though an example of classification ofpixels into three classes is described herein, it should be understoodthat embodiments are not limited thereto.

In operation 420, the iris region extracting apparatus estimates one ormore geometric parameters associated with the iris region using thegenerated classification map. For example, the iris region extractingapparatus may determine a circle, ellipse, and/or curve equation orshape to define the iris region by fitting one or more geometric models,for example, a circle, ellipse, and/or curve to the classification map.The geometric parameters may include some or all of, for example, afirst parameter of a first circle or a first ellipse corresponding to anouter boundary of the iris region, a second parameter of a second circleor a second ellipse corresponding to a boundary between the iris regionand the pupil region, a third parameter of a first curve correspondingto an upper eyelid, and a fourth parameter of a second curvecorresponding to a lower eyelid. The first parameter and the secondparameter may respectively include a coefficient to define a circle oran ellipse, such as central coordinates and a radius of the circle orfocal coordinates and lengths of a major axis and a minor axis of theellipse. The third parameter and the fourth parameter may include acoefficient to define the curve, for example.

Herein, described example circle, ellipse, and/or curve fitting examplesmay be implemented by any typical fitting operations, such as those thatconstruct a curve, or mathematical function, that has the best fit to aseries of data points, and which may also be subject to constraints,such as the aforementioned predetermined geometric properties/boundariesof a human eye and/or the boundaries between the differently classifiedregions of the eye image or the classification map. The respectivefitting operations may include interpolation, extrapolation, and/orsmoothing, for example. The fitting operations may also be based onrespectively set thresholds for example, where the fitting operationsare performed until a minimum uncertainty is present in the fittedcircle, ellipse, or curve. In addition, though geometric parameters arediscussed with respect to circles, ellipses, or curves geometricobjects/equations, embodiments are not limited thereto, and thegeometric models may be based on other geometric objects/equations. Inanother example, the fitted curves may be portions of one or moredegreed polynomial curves, for example.

In operation 430, the iris region extracting apparatus extracts the irisregion from the eye image based on the estimated geometric parameter(s).For example, the iris region extracting apparatus may define a regionsurrounded by the outer boundary of the iris region, the boundarybetween the iris region and the pupil region, and the eyelids, using thegeometric parameters, and extract or determine the defined region fromthe eye image as the iris region.

In another example, the iris region extracting apparatus may generate afinal classification map based on the geometric parameters, and extractthe iris region from the eye image using the generated finalclassification map. The iris region extracting apparatus may generatethe final classification map in which the iris region, the pupil region,and the background region are defined based on the geometric parameters,and apply the generated final classification map to the eye image toextract the iris region corresponding to the iris region in the finalclassification map.

FIG. 5 is a diagram illustrating an example iris region extractionmethod, such as the iris extraction method described with reference toFIG. 4, for example.

Referring to FIG. 5, the iris region extracting apparatus obtains aclassification map 520 from an eye image 510 using a neural networkmodel, such as any of the aforementioned neural networks or combinationof neural networks described with respect to FIG. 2. The neural networkmodel extracts pixels from the eye image 510 based on the neuralnetwork's determined or inferred respective regions for the pixels ofthe eye image, i.e., based on the previous training of the neuralnetwork, and generates or outputs the classification map 520 associatedwith an iris region based on a result of the extracting of the pixels.The classification map 520 indicates which region, for example, a pupilregion 522, an iris region 524, and a background region 526, each pixelof the eye image 510 has been classified by the neural network asbelonging to.

The iris region extracting apparatus estimates geometric parameters tobe used to define the iris region in the eye image 510 using theclassification map 520. For example, the iris region extractingapparatus fits a circular geometric model to the classification map 520,and estimates a first circle 534 corresponding to an outer boundary ofthe iris region 524 and a second circle 532 corresponding to a boundaryof the pupil region 522 (or a boundary between the pupil region 522 andthe iris region 524). According to an example, an elliptical geometricmodel, in lieu of the circular geometric model, may be used for thefitting and geometric parameters of an ellipse may be estimated. Theiris region extracting apparatus fits a curved geometric model to aboundary between an iris in an eyelid and the eyelid, and estimates afirst curve 536 corresponding to an upper eyelid and a second curve 538corresponding to a lower eyelid. A region 540 in the eye image 510 to bedefined by the first circle 534, the second circle 532, the first curve536, and the second curve 538 is extracted as the iris region of the eyeimage 510. Subsequently, an iris feature may be extracted based on theextracted region 540, and iris recognition may be performed based on theextracted iris feature.

FIG. 6 is a flowchart illustrating an example iris region extractionmethod.

Referring to FIG. 6, in operation 610, the iris region extractingapparatus generates a low-resolution image with a resolution lower thana resolution of an eye image by changing the resolution of the eyeimage, or uses select pixel information from the eye image forrepresenting a lower resolution eye image. In an example, the irisregion extracting apparatus may generate the low-resolution image of theeye image through image compression or image downsampling of the eyeimage. As a result of the image compression or the image downsampling, asize of the low-resolution image may be smaller than a size of the eyeimage.

In operation 620, the iris region extracting apparatus generates aclassification map associated with an iris region from the generatedlow-resolution image using a neural network model, such as any of theaforementioned neural networks or combination of neural networksdescribed with respect to FIG. 2. Similarly to operation 410 describedwith reference to FIG. 4, image information of the low-resolution imagemay be input to the neural network model, and the neural network modelmay extract pixels from the low-resolution image based on a determinedor inferred class or region, or classify pixels included in thelow-resolution image into the respective classes. As a result of theextracting, the classification map of the low-resolution image thatindicates which region, for example, a pupil region, an iris region, anda background region, each pixel in the low-resolution image has beenclassified by the neural network as belonging to.

In operation 630, the iris region extracting apparatus estimatesgeometric parameter(s) associated with the iris region using thegenerated classification map. In one example, the iris region extractingapparatus may adjust a size of the classification map to correspond tothe size of the eye image, and estimate the geometric parameter(s) usingthe classification map of which the size is adjusted. For example, theclassification map may be resized in a reverse manner and extent thatthe eye image had been compressed or downsampled to generate the lowerresolution eye image, such that the resized classification map has thesame dimensions of the eye image. The iris region extracting apparatusmay estimate a geometric parameter by fitting a geometric model, suchas, for example, a circle or an ellipse and a curve, to theclassification map of which the size is adjusted. In operation 640, theiris region extracting apparatus extracts the iris region from the eyeimage based on the estimated geometric parameter. For a detaileddescription of operations 630 and 640, reference may be made to thedescription of operations 420 and 430 of FIG. 4, and thus a moredetailed and repeated description of operations 630 and 640 is omittedhere. In an alternative example, the geometric model fitting may beperformed from the classification map, and the resultant fittedgeometric model adjusted to take into consideration the difference inresolutions or dimensions between the lower resolution eye image and theeye image.

FIG. 7 is a diagram illustrating an example iris region extractionmethod, such as the iris region extraction method described withreference to FIG. 6, for example.

Referring to FIG. 7, the iris region extracting apparatus generates alow-resolution image 720 by changing a resolution of an eye image 710.The iris region extracting apparatus generates a low-resolutionclassification map 730 from the generated low-resolution image 720 usinga neural network model, such as any of the aforementioned neuralnetworks or combination of neural networks described with respect toFIG. 2. Here, a low or lower resolution indicates that a resolution ofthe classification map 730 is relatively lower than the classificationmap 520 illustrated in FIG. 5, for example, which may have a same orsimilar resolution or dimensions of the eye image. Subsequently, theiris region extracting apparatus adjusts a size of the classificationmap 730 to be a size of the eye image 710 through image scaling or imageinterpolation. Thus, with the increasing of the size of theclassification map 730, the resolution of the classification map 730 maybecome identical to the resolution of the eye image 710, such asotherwise described herein. The iris region extracting apparatusdetermines geometric parameters 752, 754, 756, and 758 to define an irisregion by fitting a geometric model to a classification map 740 of theincreased size. Accordingly, the iris region extracting apparatusextracts, as the iris region from the eye image 710, a region 760defined by the geometric parameters 752, 754, 756, and 758. A process ofdetermining geometric parameters and extracting a region defined by thegeometric parameters as an iris region is described above, and thusfurther detailed and repeated descriptions of the same are omitted here.

FIG. 8 is a flowchart illustrating an example iris region extractionmethod.

According to an example, the iris region extracting apparatus mayextract an iris region from an eye image based on a refinement region.The iris region extracting apparatus may determine a plurality ofrefinement regions in the eye image using a low-resolution image of theeye image, and determine one or more geometric parameters to define theiris region based on a result of extracting pixels from the refinementregions.

Referring to FIG. 8, in operation 810, the iris region extractingapparatus generates a low-resolution image with a resolution lower thana resolution of an eye image by changing the resolution of the eyeimage, for example. Operation 810 may be identical to operation 610described with reference to FIG. 6, for example. In operation 820, theiris region extracting apparatus generates a classification mapassociated with an iris region from the generated low-resolution imageusing a first neural network model, such as any of the aforementionedneural networks or combination of neural networks described with respectto FIG. 2. Operation 820 may be identical to operation 620 describedwith reference to FIG. 6, for example. The iris region extractingapparatus may, thus, extract or classify pixels from the low-resolutionimage using the first neural network model. As a result of theextracting, coarse features of the iris region of the eye image may bedetermined. Operation 820 may be performed based on the low-resolutionimage, and thus an amount of operation or computation may be relativelysmaller than a case of performing classification directly on the entireeye image that may have a relatively high resolution. Here, operation820 may also be considered a first segmentation for the eye image.

In operation 830, the iris region extracting apparatus determines aplurality of refinement regions in the eye image using the generatedclassification map. For example, using the classification map, suchrefinement regions that may be used to perform more refinedclassifications of the eye image may be more readily determined. Arefinement region used herein refers to a partial region selected toextract a more refined feature of the eye image, and for which a higheraccuracy in classification of pixels may be needed, for example. Theiris region extracting apparatus may determine the refinement regionsbased on the classification map and geometrical structural informationpredefined with respect to the iris region. In a case of an iris,geometrical structural information may include, for example, a circle oran ellipse and a curve, which may be predefined structural information.Such predefined geometrical structural information may be informationindicating whether the shape of the iris region is to be defined by onlytwo circles respectively defining a contour of a pupil and a contour ofan iris, or information further indicating that the shape of the eye maybe further defined by a curve to define an occlusion by at least one eyelid, for example, in addition to such two circles, as only non-limitingexamples and noting that additional or alternative predefinedgeometrical structural information be predefined, e.g., before or duringthe determining of the refinement regions. In one example, locations ofthe refinement regions may be determined based on points at whichstraight lines passing through a central point of a pupil region in theclassification map and a boundary of the pupil region meet or thestraight lines and a boundary of the iris region in the classificationmap meet. Based on the geometrical structural information associatedwith a structure of the iris region, an inclination of each of thestraight lines passing through the central point of the pupil region maybe predetermined. The refinement regions may be partial regions,centered at a point at which the straight lines and the boundary of thepupil region meet or the straight lines and the boundary of the irisregion meet. A size and a shape of each of the refinement regions mayvary based on embodiment, and sizes and/or shapes of the refinementregions may be different from one another, the refinement regions may bethe same size and/or shape, or some of the refinement regions may be ofdifferent sizes and/or shapes and some refinement regions may be of thesame sizes and/or shapes.

In operation 840, the iris region extracting apparatus extracts pixelsincluded in the refinement regions using a second neural network model,such as any of the aforementioned neural networks or combination ofneural networks described with respect to FIG. 2. Here, for example,only image information of the pixels in the refinement regions may beinput to the second neural network model, and the second neural networkmodel may perform pixel-unit classification for each of these pixels.For example, the second neural network model may classify the pixels inthe refinement regions with respect to the eye image into one of a classcorresponding to an iris, a pupil, and a background. The backgroundrefers to a region that is not the iris and not the pupil. A neuralnetwork model configured to output a result of the classificationperformed for each pixel may be used as the second neural network model.As the result of the classification, a partial high-resolutionclassification map associated with the refinement regions may begenerated. Here, the classification may be performed only on therefinement regions, which are partial regions, and thus unnecessaryoperations or computation that may have been performed on other regions,e.g., if the classification was performed on the entire eye image, maybe reduced. In this example, operation 840 may be considered a secondsegmentation for the eye image.

In operation 850, the iris region extracting apparatus estimatesgeometric parameters associated with the iris region from the eye imagebased on the result of the classification obtained from operation 840.The iris region extracting apparatus may estimate the geometricparameters by fitting a geometric model to the result of theclassification performed on the refinement regions. For example, theiris region extracting apparatus may estimate a circular and/orelliptical geometric parameter, and a curved geometric parameter fromthe result of the classification performed on the refinement regionsthrough fitting of a circle and/or ellipse and a curve. In operation860, the iris region extracting apparatus extracts the iris region fromthe eye image based on the estimated geometric parameters. A process ofestimating geometric parameters and extracting an iris region from aneye image based on the estimated geometric parameters is describedabove, and thus repeated detailed descriptions are omitted here.

In one example, the iris region extracting apparatus may generate afinal classification map using geometric parameters associated with aniris region, and extract the iris region from an eye image using thegenerated final classification map. For example, when the finalclassification map is applied to the eye image, the iris regionextracting apparatus may determine, to be pixels included in the irisregion, pixels corresponding to an iris region of the finalclassification map among pixels included in the eye image. A set of thedetermined pixels may be extracted as the iris region.

As described with reference to FIG. 8, by determining a plurality ofrefinement regions, which are partial regions, using a low-resolutionimage in a first segmentation operation, and extracting an iris regionusing a result of extracting or classifying pixels in the refinementregions in a second segmentation operation, the iris region may beaccurately extracted and also an amount of operation or computationneeded to extract the iris region may be reduced.

FIG. 9 is a diagram illustrating an example iris region extractionmethod, such as the iris region extraction method described withreference to FIG. 8, for example.

Referring to FIG. 9, the iris region extracting apparatus generates alow-resolution image 915 by changing a resolution of an eye image 910.The iris region extracting apparatus generates a low-resolutionclassification map 920 from the low-resolution image 915 using a firstneural network model, such as any of the aforementioned neural networksor combination of neural networks described with respect to FIG. 2. Theclassification map 920 may include a pupil region 924 and an iris region922. Subsequently, the iris region extracting apparatus determines aplurality of refinement regions to be applied to the eye image 910 usingthe classification map 920. The refinement regions may includerefinement regions corresponding to a boundary of a pupil region, anouter boundary of an iris region, and a boundary of the iris regionoccluded by an upper eyelid or a lower eyelid, for example. The irisregion extracting apparatus determines the refinement regions using theclassification map 920 and a geometric model associated with the irisregion. The geometric model may include two circles or ellipses todefine the pupil region and the iris region, and curves to define shapesof the upper eyelid and/or the lower eyelid, for example.

In one example, refinement regions 950 corresponding to a boundary of apupil region may be determined as follows: (1) a central point of thepupil region 924 may be estimated from the classification map 920; (2)partial regions 940 around a point at which straight lines spreading atan angle preset based on a horizontal axis, for example, 0° and 90°,while passing through the central point of the pupil region 924, and theboundary of the pupil region 924 meet may be determined; and (3) regionsin the eye image 910 corresponding to locations of the partial regions940 of the classification map 920 may be determined to be the refinementregions 950 corresponding to the boundary of the pupil region.

In an example, refinement regions 942 and 944 corresponding to an outerboundary of an iris region may be determined as follows: (1) partialregions 932 and 934 around a point at which straight lines spreading atan angle preset based on the horizontal axis, for example, 0°, 15°, and165° counterclockwise, while passing through the central point of thepupil region 924, and a boundary of the iris region 922 meet may bedetermined; and (2) regions in the eye image 910 corresponding tolocations of the partial regions 932 and 934 of the classification map920 may be determined to be the refinement regions 942 and 944corresponding to the outer boundary of the iris region.

In an example, refinement regions 946 and 948 corresponding to aboundary of an iris region occluded by an upper eyelid and a lowereyelid may be determined as follows: (1) partial regions 936 and 938around a point at which straight lines spreading at an angle presetbased on the horizontal axis, for example, 75°, 90°, and 115°, whilepassing through the central point of the pupil region 924, and theboundary of the iris region 922 meet may be determined; and (2) regionsin the eye image 910 corresponding to locations of the partial regions936 and 938 of the classification map 920 may be determined to be therefinement regions 946 and 948 corresponding to the boundary of the irisregion occluded by the upper eyelid and the lower eyelid.

Accordingly, in an example, the iris region extracting apparatus mayextract or classify pixels included in the refinement regions 942, 944,946, 948, and 950 of the eye image 910 using a second neural networkmodel, such as any of the aforementioned neural networks or combinationof neural networks described with respect to FIG. 2 and as discussedabove with respect to FIG. 8. The second neural network model mayclassify the pixels more specifically than the first neural networkmodel. For example, there may be more pixel information available toprovide insight to the classification operation due to the use the eyeimage 910, than previously available for classification of pixels of thelow-resolution image 915, and the second neural network model may beconfigured to perform more precise classification determinations orinferences. As a result of the extracting or the classifying by thesecond neural network model, a high-resolution classification map 960indicating which region, for example, of a pupil, an iris, and abackground, each pixel in the refinement regions 942, 944, 946, 948, and950 belongs to may be generated. The high resolution term used hereinmay indicate that a resolution of the classification map 960 isrelatively higher than the resolution of the classification map 920.

The iris region extracting apparatus estimates geometric parameters todefine an iris region from pixel classification results 962, 964, 966,968, and 970 indicated in the classification map 960. The iris regionextracting apparatus estimates a geometric parameter 978 to define apupil region from the pixel classification result 970 using a circle orellipse fitting method, and estimates a geometric parameter 976 todefine an outer boundary of an iris from the pixel classificationresults 962 and 964. In addition, the iris region extracting apparatusestimates a geometric parameter 972 to define a boundary of an irisregion occluded by an upper eyelid from the pixel classification result966 using a curve fitting method. The iris region extracting apparatusestimates a geometric parameter 974 to define a boundary of an irisregion occluded by a lower eyelid from the pixel classification result968.

According to an example, the iris region extracting apparatus mayselectively generate a classification map 980 based on the estimatedgeometric parameters 972, 974, 976, and 978. The classification map 980may include a pupil region 982 defined by the geometric parameter 978,and an iris region 984 defined by the geometric parameters 972, 974,976, and 978.

The iris region extracting apparatus extracts an iris region 992 fromthe eye image 910 using the geometric parameters 972, 974, 976, and 978,or the classification map 980. The iris region extracting apparatusextracts, as the iris region 992 from the eye image 910, a region in theeye image 910 defined by the geometric parameters 972, 974, 976, and978, or a region corresponding to the iris region 984 of theclassification map 980.

As described above, the iris region extracting apparatus may performhighly accurate classification on partial regions in an eye image toextract an iris region from the eye image. Thus, the iris regionextracting apparatus may extract the iris region more accurately andrapidly. In addition, the iris region extracting apparatus may reduceresources needed to extract the iris region. Still further, the irisregion extracting apparatus may then compare features of the extractediris region 992 to features of registered iris regions, and approve orreject an authentication of the eye image based on a result of thatcomparison.

FIG. 10 is a flowchart illustrating an example iris region extractionmethod.

Referring to FIG. 10, in operation 1010, the iris region extractingapparatus generates a low-resolution image with a resolution lower thana resolution of an eye image by changing the resolution of the eyeimage. In operation 1020, the iris region extracting apparatus obtains afirst geometric parameter associated with an iris region from thegenerated low-resolution image using a first neural network model, suchas any of the aforementioned neural networks or combination of neuralnetworks described with respect to FIG. 2. For example, when imageinformation of the low-resolution image is input to the first neuralnetwork model, the first neural network model may obtain the firstgeometric parameter associated with a geometric structure of the irisregion in the low-resolution image. The first geometric parameter mayinclude information associated with a circle or an ellipse and a curveto define a shape of each of a pupil and an iris. The first neuralnetwork model may be a neural network model pre-trained to outputgeometric parameters associated with the iris region based on the inputimage information.

In operation 1030, the iris region extracting apparatus determines aplurality of refinement regions in the eye image using the firstgeometric parameter. The iris region extracting apparatus may determinea coarse iris region and a coarse pupil region from the eye image usingthe first geometric parameter, and determine refinement regions based onpredefined geometrical structural information. In one example, locationsof the refinement regions may be determined based on points at whichstraight lines passing through a central point of the coarse pupilregion determined in the eye image and a boundary of the coarse pupilregion meet or the straight lines and a boundary of the coarse irisregion determined in the eye image meet. The refinement regions may bepartial regions, centered at a point at which the straight lines and theboundary of the pupil region or the iris region meet, for example. Asize and a shape of each of the refinement regions may vary based onembodiment, and sizes and/or shape of the refinement regions may bedifferent from one another, they may be the same size and/or shape, orsome of the refinement regions may be of different sizes and/or shapesand some refinement regions may be of the same sizes and/or shapes.Dissimilarly to the example illustrated in FIG. 6, the iris regionextracting apparatus may determine the refinement regions without usinga classification map, for example.

In operation 1040, the iris region extracting apparatus extracts pixelsincluded in the refinement regions using a second neural network model,such as any of the aforementioned neural networks or combination ofneural networks described with respect to FIG. 2. In operation 1050, theiris region extracting apparatus estimates a second geometric parameterassociated with the iris region from the eye image based on a result ofthe extracting performed in operation 1040. In operation 1060, the irisregion extracting apparatus extracts the iris region from the eye imagebased on the estimated second geometric parameter. Operations 1040through 1060 described with reference to FIG. 10 may correspond tooperations 840 and 860 described with reference to FIG. 8, and thus amore detailed and repeated description of operations 1040 through 1060is omitted here.

FIG. 11 is a diagram illustrating an example apparatus that isconfigured to perform iris region extraction according to one or more orall methods described herein. As illustrated in FIG. 11, an iris regionextracting apparatus 1100 may extract an iris region from an input imageincluding an eye region. Referring to FIG. 11, the iris regionextracting apparatus 1100 includes a processor 1110 and a memory 1120,for example.

The processor 1110 may perform one or more or all operations describedwith reference to FIGS. 1 through 10. For example, the processor 1110may be configured to obtain a classification map and one or moregeometric parameters associated with the iris region from the inputimage using a neural network model, such as any of the aforementionedneural networks or combination of neural networks described with respectto FIG. 2 which may be stored in the memory, for example, and mayidentify the iris region from the input image using the obtainedclassification map and the obtained geometric parameter(s). Theprocessor 1110 may extract the identified iris region, and then performauthentication of the extracted iris region, e.g., by comparing featuresof the extracted iris region to features of reference or registered irisregion(s) stored in the memory, for example. For example, the processor1110 may determine an iris feature of the extracted iris region, anddetermine whether to authenticate a user based on whether the determinediris feature matches a registered feature. The processor 1110 may beembodied as an array of a plurality of logic gates or alternativelyembodied as computing hardware in other forms. The processor 1110 isrepresentative of one or more processors.

The memory 1120 is one or more non-transitory computer readable mediathat may store instructions, such that when executed by the processor1110, cause the processor 1110 to perform one or more or all operationsdescribed with reference to FIGS. 1 through 10, store a result ofcalculations performed by the iris region extracting apparatus 1100, andstore one or more of the aforementioned neural networks. The memory 1120may include, for example, a high-speed random access memory (RAM), and anonvolatile computer readable storage medium, for example, at least onedisk storage device, a flash memory device, and other nonvolatile solidstate memory devices.

FIG. 12 is a diagram illustrating an example computing apparatusconfigured to perform iris region extraction according to one or more orall methods described herein.

As illustrated in FIG. 12, a computing apparatus 1200 may obtain animage that includes an eye region of a user, and extract an iris regionfrom the obtained image. The computing apparatus 1200 may first performa region of interest (ROI) or cropping of the obtained image beforeperforming the iris region extraction, such as to focus the iris regionextraction on a resultant image that is primarily or only of one or moreeyes, e.g., considering a desired resolution or set dimensions of imageinput layers of the applied neural network model. The computingapparatus 1200 may extract an iris feature from the extracted irisregion, and perform user authentication based on the extracted irisfeature. The computing apparatus 1200 may be configured to perform irisextraction operations, such as the iris extraction operations of theiris region extracting apparatus 1100 illustrated in FIG. 11. Referringto FIG. 12, the computing apparatus 1200 includes a processor 1210, amemory 1220, a first camera 1230, a second camera 1235, a storage device1240, an input device 1250, an output device 1260, a network device1270, an optical source 1275, and a bus 1280. Each component of thecomputing apparatus 1200 may exchange data with other components throughthe bus 1280.

The processor 1210 may perform various functions and instructions to beimplemented in the computing apparatus 1200. For example, the processor1210 may process instructions stored in the memory 1220 or the storagedevice 1240. The processor 1210 may perform one or more or alloperations described above with reference to FIGS. 1 through 11, forexample.

The memory 1220 is one or more non-transitory computer readable mediathat may store information in the computing apparatus 1200. The memory1220 may include, for example, a computer readable storage medium or acomputer readable storage device. The memory 1220 may include, forexample, a RAM, a dynamic RAM (DRAM), and a static RAM (SRAM), and anonvolatile memory in other forms well-known in the relevant art. Thememory 1220 may store the instructions to be implemented by theprocessor 1210, and store related information during software or anapplication being executed by the computing apparatus 1200, includinginstructions, that when executed by the processor 1210, cause theprocessor to implement one or more or all operations described abovewith respect to FIGS. 1-11. In addition, either or both of the memory1220 and the storage device 1240 may store the aforementioned neuralnetwork models described herein.

The first camera 1230 may capture a still image, a video image, or bothof the images. In one example, the first camera 1230 may capture animage input from a user making an attempt at user authentication. Thesecond camera 1235 may capture an infrared image. An infrared ray may beradiated from the optical source 1275, e.g., which radiates an infraredlight externally of the computing apparatus 1200, and an infrared rayreflected by the user making an attempt at iris verification may becaptured by the second camera 1235.

According to an example, the computing apparatus 1200 may include atleast one of the first camera 1230 or the second camera 1235, e.g., atleast one of the first camera 1230, only the first camera 1230, at leastone of the second camera 1235, only the second camera 1235, or one ormore first cameras 1230 and one or more second cameras 1235. Thecomputing apparatus 1200 may include further cameras. In one example,the computing apparatus 1200 may select an image including a clearer eyeregion of the user from a first image obtained by the first camera 1230and a second image obtained by the second camera 1235, and perform irisextraction using only the selected image. For example, the computingapparatus 1200 may select the image from which the iris region is to beextracted based on a determined quality of each of the first image andthe second image, and a determined presence of an artifact, for example,a light blur, in the first image and the second image.

The storage device 1240 is one or more non-transitory computer readablemedia that may include a computer readable storage medium or a computerreadable storage device. In one example, the storage device 1240 maystore a great amount of information compared to the memory 1220, andstore the information for a long period of time. The storage device 1240may include, for example, a magnetic hard disk, an optical disc, a flashmemory, an erasable programmable read-only memory (EPROM), a floppydisk, and a nonvolatile memory in other forms well-known in the relevantart. The storage device 1240 may also store the instructions to beimplemented by the processor 1210, and store related information duringsoftware or an application being executed by the computing apparatus1200, including instructions, that when executed by the processor 1210,cause the processor to implement one or more or all operations describedabove with respect to FIGS. 1-11.

The input device 1250 may receive an input, for example, a tactileinput, a video input, an audio input, and a touch input, from the user.The input device 1250 may include, for example, a keyboard, a mouse, atouchscreen, a microphone, and another device configured to detect theinput from the user and transfer the detected input to the computingapparatus 1200.

The output device 1260 may provide an output of the computing apparatus1200 to the user through a visual, auditory, or tactile method. Theoutput device 1260 may include, for example, a liquid crystal display(LCD), a light-emitting diode (LED) display, a touchscreen, a speaker,an oscillator, and another device configured to provide the output tothe user. The output device may also include respective outputinterfaces for external displays, touchscreens, speakers, oscillators,or to other components of a larger or external processing device, suchas a device that is provided the extracted iris region and performs anauthentication on the same. The output device may also indicate successof an authentication of an extracted iris with a registered iris orfeatures of the same, for example.

The network device 1270 is a hardware module configured to communicatewith external devices through a wired and/or wireless networks. Thenetwork device 1270 may include, for example, an Ethernet card, anoptical transceiver, and a radio frequency (RF) transceiver, or anothernetwork interface card configured to transmit and receive information.The network device 1270 may be configured to communicate with theexternal device using a communication method, for example, Bluetooth,WiFi, a third generation (3G) method, a long term evolution (LTE)method, an a fifth generation (5G) method, as only examples. In anexample, in cooperation with control by the processor 1210 and using oneor both of the memory 1220 and storage device 1240, the network device1270 may also receive and store updates or changes to any one or more orall neural networks described herein, including models and instructionsfor implementing the same through other machine learning models, whichare implemented as further embodiments. The network device 1270 may befurther configured to provide or transmit an extracted iris region,e.g., according to any of the iris extraction methods described herein,to another processing device that may be remote and for the otherprocessing device's performance of an authentication operation based onthe transmitted or provided extracted iris region.

The computing apparatuses, iris region extracting apparatuses, computingapparatus 110, iris region extracting apparatus 1100, processor 1110,memory 1120, computing apparatus 1200, processor 1210, memory 1220,first camera 1230, second camera 1235, storage device 1240, input device1250, output device 1260, network device 1270, optical source 1275, asonly examples, described with respect to FIGS. 1-12 and that perform theoperations described in this application are implemented by hardwarecomponents configured to perform the operations described in thisapplication that are performed by the hardware components. Examples ofhardware components that may be used to perform the operations describedin this application where appropriate include controllers, sensors,generators, drivers, memories, comparators, arithmetic logic units,adders, subtractors, multipliers, dividers, integrators, and any otherelectronic components configured to perform the operations described inthis application. In other examples, one or more of the hardwarecomponents that perform the operations described in this application areimplemented by computing hardware, for example, by one or moreprocessors or computers. A processor or computer may be implemented byone or more processing elements, such as an array of logic gates, acontroller and an arithmetic logic unit, a digital signal processor, amicrocomputer, a programmable logic controller, a field-programmablegate array, a programmable logic array, a microprocessor, or any otherdevice or combination of devices that is configured to respond to andexecute instructions in a defined manner to achieve a desired result. Inone example, a processor or computer includes, or is connected to, oneor more memories storing instructions or software that are executed bythe processor or computer. Hardware components implemented by aprocessor or computer may execute instructions or software, such as anoperating system (OS) and one or more software applications that run onthe OS, to perform the operations described in this application. Thehardware components may also access, manipulate, process, create, andstore data in response to execution of the instructions or software. Forsimplicity, the singular term “processor” or “computer” may be used inthe description of the examples described in this application, but inother examples multiple processors or computers may be used, or aprocessor or computer may include multiple processing elements, ormultiple types of processing elements, or both. For example, a singlehardware component or two or more hardware components may be implementedby a single processor, or two or more processors, or a processor and acontroller. One or more hardware components may be implemented by one ormore processors, or a processor and a controller, and one or more otherhardware components may be implemented by one or more other processors,or another processor and another controller. One or more processors, ora processor and a controller, may implement a single hardware component,or two or more hardware components. A hardware component may have anyone or more of different processing configurations, examples of whichinclude a single processor, independent processors, parallel processors,single-instruction single-data (SISD) multiprocessing,single-instruction multiple-data (SIMD) multiprocessing,multiple-instruction single-data (MISD) multiprocessing, andmultiple-instruction multiple-data (MIMD) multiprocessing.

The methods illustrated in FIGS. 1-12 that perform the operationsdescribed in this application are performed by computing hardware, forexample, by one or more processors or computers, implemented asdescribed above executing instructions or software to perform theoperations described in this application that are performed by themethods. For example, a single operation or two or more operations maybe performed by a single processor, or two or more processors, or aprocessor and a controller. One or more operations may be performed byone or more processors, or a processor and a controller, and one or moreother operations may be performed by one or more other processors, oranother processor and another controller. One or more processors, or aprocessor and a controller, may perform a single operation, or two ormore operations.

Instructions or software to control computing hardware, for example, oneor more processors or computers, to implement the hardware componentsand perform the methods as described above may be written as computerprograms, code segments, instructions or any combination thereof, forindividually or collectively instructing or configuring the one or moreprocessors or computers to operate as a machine or special-purposecomputer to perform the operations that are performed by the hardwarecomponents and the methods as described above. In one example, theinstructions or software include machine code that is directly executedby the one or more processors or computers, such as machine codeproduced by a compiler. In another example, the instructions or softwareincludes higher-level code that is executed by the one or moreprocessors or computer using an interpreter. The instructions orsoftware may be written using any programming language based on theblock diagrams and the flow charts illustrated in the drawings and thecorresponding descriptions in the specification, which disclosealgorithms for performing the operations that are performed by thehardware components and the methods as described above.

The instructions or software to control computing hardware, for example,one or more processors or computers, to implement the hardwarecomponents and perform the methods as described above, and anyassociated data, data files, and data structures, may be recorded,stored, or fixed in or on one or more non-transitory computer-readablestorage media. Examples of a non-transitory computer-readable storagemedium include read-only memory (ROM), random-access memory (RAM), flashmemory, CD-ROMs, CD-Rs, CD+Rs, CD-RWs, CD+RWs, DVD-ROMs, DVD-Rs, DVD+Rs,DVD-RWs, DVD+RWs, DVD-RAMs, BD-ROMs, BD-Rs, BD-R LTHs, BD-REs, magnetictapes, floppy disks, magneto-optical data storage devices, optical datastorage devices, hard disks, solid-state disks, and any other devicethat is configured to store the instructions or software and anyassociated data, data files, and data structures in a non-transitorymanner and provide the instructions or software and any associated data,data files, and data structures to one or more processors or computersso that the one or more processors or computers can execute theinstructions. In one example, the instructions or software and anyassociated data, data files, and data structures are distributed overnetwork-coupled computer systems so that the instructions and softwareand any associated data, data files, and data structures are stored,accessed, and executed in a distributed fashion by the one or moreprocessors or computers.

While this disclosure includes specific examples, it will be apparentafter an understanding of the disclosure of this application thatvarious changes in form and details may be made in these exampleswithout departing from the spirit and scope of the claims and theirequivalents. The examples described herein are to be considered in adescriptive sense only, and not for purposes of limitation. Descriptionsof features or aspects in each example are to be considered as beingapplicable to similar features or aspects in other examples. Suitableresults may be achieved if the described techniques are performed in adifferent order, and/or if components in a described system,architecture, device, or circuit are combined in a different manner,and/or replaced or supplemented by other components or theirequivalents. Therefore, the scope of the disclosure is defined not bythe detailed description, but by the claims and their equivalents, andall variations within the scope of the claims and their equivalents areto be construed as being included in the disclosure.

What is claimed is:
 1. A processor implemented iris region extractionmethod, the method comprising: obtaining an image that includes an eyeand performing a cropping of the image to obtain an eye image;extracting an iris region from the obtained eye image using a trainedneural network model; extracting features from the extracted irisregion; and matching the extracted features to registered irisinformation.
 2. The method of claim 1, wherein the extracting of theiris region comprises: extracting pixels corresponding to the irisregion from the eye image as an output of the trained neural networkmodel.
 3. The method of claim 1, wherein the extracting of the irisregion comprises: generating a lower-resolution image of the eye imageby changing a resolution of the eye image; generating a classificationmap associated with the iris region from the lower-resolution imageusing the trained neural network model; estimating one or more geometricparameters associated with the iris region using the generatedclassification map; and extracting the iris region from the eye imagebased on the estimated one or more geometric parameters.
 4. The methodof claim 3, wherein the estimating of the one or more geometricparameters comprises: adjusting a size of the classification map tomatch a size of the eye image; and estimating the one or more geometricparameters using the adjusted size classification map.
 5. The method ofclaim 1, wherein the extracting of the iris region comprises: generatinga lower-resolution image of the eye image; generating a classificationmap associated with the iris region from the generated lower-resolutionimage using a first trained neural network model; determining aplurality of refinement regions in the eye image using the generatedclassification map; extracting pixels from the refinement regions usinga second trained neural network; and extracting the iris region from theeye image based on a result of the extracting of the pixels.
 6. Themethod of claim 5, wherein the determining of the refinement regionscomprises: determining the refinement regions in the eye image based onthe classification map and structural information predefined withrespect to iris regions.
 7. The method of claim 5, wherein theextracting of the iris region comprises: estimating one or moregeometric parameters associated with the iris region based on the resultof the extracting of the pixels; and extracting the iris region from theeye image based on the estimated one or more geometric parameters. 8.The method of claim 1, wherein the extracting of the iris regioncomprises: generating a lower-resolution image of the eye image;obtaining a geometric parameter associated with the iris region from thegenerated lower-resolution image using a first trained neural networkmodel; determining a plurality of refinement regions in the eye imageusing the obtained geometric parameter; extracting pixels from therefinement regions using a second trained neural network model; andextracting the iris region from the eye image based on a result of theextracting of the pixels.
 9. The method of claim 8, wherein theextracting of the iris region comprises: estimating one or moregeometric parameters associated with the iris region based on the resultof the extracting of the pixels; and extracting the iris region from theeye image based on the estimated one or more geometric parameters. 10.The method of claim 1, wherein the extracting of the iris regioncomprises: obtaining a geometric parameter associated with the irisregion from the eye image using the trained neural network model; andextracting the iris region from the eye image based on the obtainedgeometric parameter.
 11. The method of claim 1, wherein the obtaining ofthe eye image comprises: extracting a region of interest (ROI),including the iris region, from the input image, as the obtained eyeimage.
 12. A non-transitory computer-readable storage medium storinginstructions that, when executed by a processor, cause the processor toperform the method of claim
 1. 13. A processor-implemented iris regionextraction method, the method comprising: obtaining an eye image; andextracting an iris region from the obtained eye image using a trainedneural network model, wherein the extracting of the iris regioncomprises: performing a first segmentation of the eye image by providinga lower resolution image of the eye image to a first trained neuralnetwork; performing a second segmentation of the eye image using asecond trained neural network, the second segmentation of the eye imagebeing dependent on results of the first segmentation of the eye image;and extracting the iris region from the eye image based on results ofthe second segmentation.
 14. A processor-implemented iris regionextraction method, the method comprising: obtaining an eye image; andextracting an iris region from the obtained eye image using a trainedneural network model, wherein the extracting of the iris regioncomprises: generating a first classification map associated with theiris region from the eye image using the trained neural network model;estimating one or more geometric parameters associated with the irisregion using the generated first classification map; and extracting theiris region from the eye image based on the estimated one or moregeometric parameters.
 15. The method of claim 14, wherein the estimatingof the one or more geometric parameters includes performing a fittingoperation of one or more geometric equations to the first classificationmap, the fitting including estimating geometric parameters of at leastone of a circle, ellipse, or a curve that are fitted to features of thefirst classification map.
 16. The method of claim 15, wherein the one ormore geometric parameters include plural geometric parameters, includinga first parameter to define a shape of a first circle or a first ellipsecorresponding to an outer boundary of the iris region represented in thefirst classification map and a second parameter to define a shape of asecond circle or a second ellipse corresponding to a boundary betweenthe iris region and a pupil region represented in the firstclassification map.
 17. The method of claim 15, wherein the pluralgeometric parameters further include at least one of a third parameterto define a shape of a first curve corresponding to an upper eyelid or afourth parameter to define a shape of a second curve corresponding to alower eyelid.
 18. The method of claim 14, wherein the extracting of theiris region comprises: generating a second classification map based onthe estimated one or more geometric parameters; and extracting the irisregion from the eye image using the generated second classification map.19. A processor-implemented iris region extraction method, the methodcomprising: obtaining an eye image; and extracting an iris region fromthe obtained eye image using a trained neural network model, furthercomprising: respectively analyzing a first eye image, which is acaptured color image of an eye of a user, and a second eye image, whichis a captured infrared image of the eye of the user, to select one ofthe first eye image and the second eye image to be the obtained eyeimage.
 20. An apparatus, comprising: one or more processors configuredto: obtain an image that includes an eye and perform a cropping of theimage to obtain an eye image; extract an iris region from the obtainedeye image using a trained neural network model; extract features fromthe extracted iris region; and match the extracted features toregistered iris information.
 21. An apparatus, comprising: one or moreprocessors configured to extract an iris region from an obtained eyeimage using a trained neural network model, wherein the one or moreprocessors are configured to: generate a classification map associatedwith the iris region from the eye image using the trained neural networkmodel; estimate one or more geometric parameters associated with theiris region using the generated classification map; and extract the irisregion from the eye image based on the estimated one or more geometricparameters.
 22. The apparatus of claim 21, wherein, to estimate the oneor more geometric parameters, the one or more processors are configuredto perform a fitting operation of one or more geometric equations to theclassification map, the fitting including estimating geometricparameters of at least one of a circle, ellipse, or a curve that arefitted to features of the classification map.
 23. The apparatus of claim22 wherein the one or more geometric parameters include plural geometricparameters, including a first parameter to define a shape of a firstcircle or a first ellipse corresponding to an outer boundary of the irisregion represented in the classification map and a second parameter todefine a shape of a second circle or a second ellipse corresponding to aboundary between the iris region and a pupil region represented in theclassification map.
 24. An apparatus, comprising: one or more processorsconfigured to extract an iris region from an obtained eye image using atrained neural network model, wherein the one or more processors areconfigured to: generate a lower-resolution image of the eye image;generate a classification map associated with the iris region from thegenerated lower-resolution image using a first trained neural networkmodel, with the classification map having a lower resolution than theeye image; determine a plurality of refinement regions in the eye imageusing the generated classification map; extract pixels from therefinement regions using a second trained neural network model; andextract the iris region from the eye image based on a result of theextracting of the pixels.
 25. An apparatus, comprising: one or moreprocessors configured to extract an iris region from an obtained eyeimage using a trained neural network model, wherein the one or moreprocessors are configured to: generate a lower-resolution image of theeye image; obtain a geometric parameter associated with the iris regionfrom the generated lower-resolution image using a first trained neuralnetwork model; determine a plurality of refinement regions in the eyeimage using the obtained geometric parameter; extract pixels from thedetermined plural refinement regions using a second trained neuralnetwork model; estimate one or more geometric parameters associated withthe iris region based on a result of the extracting of the pixels; andextract the iris region from the eye image based on the estimated one ormore geometric parameters.
 26. A processor implemented iris regionextraction method, the method comprising: providing a first image for aneye to a first trained neural network to generate an output of the firsttrained neural network; estimating one or more geometric parameters byperforming a fitting operation of one or more geometric equations usingthe output of the first trained neural network, the fitting includingestimating geometric parameters of at least one of a circle, ellipse, ora curve for an iris region of the eye; and extracting the iris regionfrom a second image for the eye based on the estimated one or moregeometric parameters.
 27. The method of claim 26, wherein the extractingof the iris region comprises: determining a plurality of refinementregions in the second image based on the estimated one or more geometricparameters; extracting pixels from the refinement regions using a secondneural network model; and extracting the iris region from the secondimage based on a result of the extracting of the pixels.
 28. The methodof claim 27, wherein the extracting of the iris region comprises:estimating at least one geometric parameter associated with the irisregion based on the result of the extracting of the pixels; andextracting the iris region from the second image based on the estimatedat least one geometric parameter.
 29. The method of claim 26, whereinthe one or more geometric parameters include plural geometricparameters, including a first parameter to define a shape of a firstcircle or a first ellipse corresponding to an outer boundary of the irisregion represented in the output of the first trained neural network anda second parameter to define a shape of a second circle or a secondellipse corresponding to a boundary between the iris region and a pupilregion represented in the output of the first trained neural network.30. The method of claim 26, wherein the first image is an infrared imageand the second image is a color image.